Transparent deferred spooling store and forward based on standard network system and client interface

ABSTRACT

A transparent batch file transfer is provided from a client to a server via a batch pool system. The batch pool system may be implemented by a proxy file server which is used to receive the file transfer from the client device and free the client device as soon as possible. The file transfer to an intended remote server is carried out by a batch transfer system at the proxy file server. The user of the client machine may then use their device to perform other tasks while the file transfer is completed by the proxy file server batch transfer system. The file transfer is coordinated by a background transfer module that is integrated with file system protocols. Hence, there is no new system or software for a user of the client to learn or operate.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation and claims the prioritybenefit of U.S. application Ser. No. 14/556,157, issuing as U.S. Pat.No. 9,917,882, the disclosure of which is incorporated herein byreference.

BACKGROUND

As businesses utilization of network and “cloud” solutions for digitalstorage and processing increases, the performance required from thenetwork and cloud solutions has increased as well. For instance,business operations may require storage or processing of files on acentral file server, and as a pre-requisite may require collation offiles on the central file server. Files may be transferred to the serverusing network file server protocols optimized for a LAN (local areanetwork).

As a business grows geographically, some of these files may originate atremote locations with poor network connectivity and so uploading offiles using LAN optimized file sharing protocols can be slow.

The slow upload performance can restrict the activities of personnel whomust upload the files or whose equipment is in use until the transfer iscompleted and therefore have negative consequences for the efficiency ofthe business.

There is a need to reduce the time spent performing uploads so thatpersonnel and equipment can be released for other activities, even whilethe transfer is still in progress.

SUMMARY

The present technology, roughly described, provides for faster filetransfers from a client to a server via a batch pool system. The batchpool system may be implemented by a proxy file server which is used toreceive the file transfer from the client device and free the clientdevice as soon as possible. The file transfer to an intended remoteserver is carried out by a batch transfer system at the proxy fileserver. The user of the client machine may then use their device toperform other tasks while the file transfer is completed by the proxyfile server batch transfer system. The file transfer is coordinated by abackground transfer module that is integrated with a client interface.Hence, there is no new system or software for a user of the client tolearn or operate.

In an embodiment, a method for transferring files may receive a firstfile data from a client by a first server. The first file data intendedfor a second server. A data acknowledgment may be transmitted to theclient from first server. A second file data may be received from theclient. An indication that transfer is complete may be transmitted tothe client by the first server. The first file data and second file datamay be transmitted to the second server. A close file request may betransmitted to the second server from the first server.

A system for tracing a distributed transaction may include a processor,memory and one or more modules stored in the memory. The one or moremodules may be executable by the processor to receive a first file datafrom client by first server, the first file data intended for a secondserver, transmit data acknowledgment to client from first server,receive a second file data from the client, transmit an indication thattransfer is complete to the client by the first server, transmit thefirst file data and second file data to the second server, and transmita close file request to the second server from the first server.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for performing a batch filetransfer

FIG. 2 is a block diagram of a proxy file server.

FIG. 3 is a method for performing a background file transfer.

FIG. 4 is a method identifying a suitable file for background filetransfer.

FIG. 5 is a method for managing a background file transfer.

FIG. 6 is a block diagram of a computing system for implementing thepresent technology.

DETAILED DESCRIPTION

The present technology, roughly described, provides for faster filetransfers from a client to a server via a batch pool system. The batchpool system may be implemented by a proxy file server which is used toreceive the file transfer from the client device and free the clientdevice as soon as possible. The file transfer to an intended remoteserver is carried out by a batch transfer system at the proxy fileserver. The user of the client machine may then use their device toperform other tasks while the file transfer is completed by the proxyfile server batch transfer system. The file transfer is coordinated by abackground transfer module that is integrated with file systemprotocols. Hence, there is no new system or software for a user of theclient to learn or operate.

The spooler permits faster transfer from the client, but not fastertransfer to the server. The batch spooler will continue to transfer tothe server at the network speed after the client has disconnected. Thepresent technology allows the user to see a standard interface or viewof a remote file server and transparently benefit from batch spoolerfeatures.

In a standard batch system, the spooler is generally local to the, andthe client can be considered to transfer a file to the spooler, and thenanother file to the spooler, and so on. The present technologyintegrates with a network file system protocol (for example, as aproxy). In this perspective, the client sends a portion of a file, whichis acknowledged, and then another portion, which is acknowledged, andfinally after the last portion is acknowledged, the file is closed. Ifthe network connection is slow, it will take a lot of time until anintended destination server acknowledges the last portion, because eachportion must traverse the slow network. The present technology has theproxy acknowledge each portion of the file so the client will send thenext portion immediately, before the early acknowledged data has evenreached the file server. By acknowledging all data immediately, theclient will transfer the whole file and then close the file. The presenttechnology may the close as if it were really closed on the server,while in reality we continue to transfer the file to the server.

While the client supposes to write a file to the server without batchspooling, the present technology causes it to spool the file to use sothat we can transfer to the server. Thus the client transparently getsthe benefit of a spooling system which is a fast initial transfer. Asystem administrator can add spooling using this method without alteringthe user's workflow or behavior.

FIG. 1 is a block diagram of a system for performing a batch filetransfer. FIG. 1 includes a client 110, network 120, proxy file server130, network 140, and server 150. A client may perform a batch filetransfer with server 150, uploading a batch of files to the server.Network 140 may be a poor performing network, however, which may causethe file transfer to take an undesireably long time. Proxy file server130 is used to implement a batch spooler transparently, and may beplaced close to client 110 and receive the batch files via network 120.The proxy file server may receive the batch of files, freeing up client110 from the batch file transfer, and may then upload the files to theserver 150 automatically. Proxy file server 130 is discussed in moredetail below with respect to FIG. 2.

FIG. 2 is a block diagram of a proxy file server 200. Proxy file server200 may include a proxy application 210 and background transfer module220. Proxy file server 210 may provide an interface to client 110 formanaging a batch file transfer between client 110 and server 150. Theinterface may be an administrative interface rather than a userinterface, and may allow the user to provide input regarding what filesor file types to transfer, when to transfer, and receive other inputfrom the client device user. The interface may also provide the userwith information regarding the batch file transfer, such as provide thestatus of a transfer, the progress of the transfer, and otherinformation. The proxy application 210 may also perform the high levelmanagement of the transfer, detect when the transfer is complete, and soforth.

Background transfer module 220 may integrate with proxy application 210to improve file transfer performance. Background transfer module 220 maybe implemented within proxy application 210, for example as a plug-in,or work in conjunction with proxy application 210.

Module 220 may operate at a protocol level to intercept protocolmessages and data from the client as well as send messages to theclient. For example, module 220 may send acknowledgement messages inresponse to receiving a file or portion of a file of the batch files. Insome instances, the acknowledgement may be sent immediately to initiatethe next write request for a file portion as soon as possible. When fileportions are received, they may be placed in a buffer until the proxyapplication can upload them to the server.

When a file transfer is complete from the client side, the client maysend a close request. Module 220 may provide a close message, such as aclose acknowledgment, back to the client to release the client from thebatch file transfer session, even though the files are still beinguploaded to the server. The file transfer session will not be closedwith the server until the batch files have all been uploaded to theserver.

FIG. 3 is a method for performing a background file transfer. Filetransfer preferences are configured at step 305. File transferpreferences may include specifying what type of file may be suitable fora background file transfer. The preferences may include a particularserver, share or folder, a file extension, a filename, and IO operationpatterns. The preferences may be set by a user or an administratorthrough the proxy application 210.

A user session is started at step 310. The user session may includehaving a user connect to the proxy application from an application onthe user client device. A file transfer request is received by the proxyserver from the client at step 315. The file transfer request mayindicate the files, file locations, and other data associated with abatch of files to transfer from the client to a particular server. Adetermination is made as to whether the files are suitable forbackground file transfer at step 320. In some instances, thedetermination may be made later, for example after a file or portion ofthe file has already been transferred. The file may be suitable if itsatisfies one or more preferences for background file transfer.Determining whether a file is suitable for background transfer isdiscussed in more detail below with respect to FIG. 4. If the filesubject to the file transfer request is not suitable for background filetransfer, the file is not uploaded to the server via background filetransfer at step 325 and the operation of the method continues to step315. The file may be uploaded via another means of uploading or the filetransfer may be denied.

If the file is suitable for background transfer, the file data, such asfor example a portion of the file, may be received by proxy server 130at step 330. The file transfer of the batch of files commences, with thefirst file or file portion received at step 330. A transmitacknowledgment is transmitted to the client at step 335. Theacknowledgment is sent by the background transfer module immediatelyafter receiving the first file or file portion. As such, theacknowledgment is sent before the file is received and stored at theintended destination—server 150. By providing the acknowledgmentimmediately, the client may then proceed to send the next batch file orfile portion, thereby expediting the time to receive the files from theclient by the proxy server.

The received data is buffered at step 340. The buffered data may be sentto the intended recipient—server 150—at step 345. The time to send thebuffered data to the server by the proxy server will take longer thanthe time to receive the data from the client by the proxy server. Insome instances, files or file portions may be received from the clientand stored in the buffer, with acknowledgements sent immediately to theclient, until there are no further files or portions in the batch. Thefiles or file portions in the buffer may be sent to the server inparallel with the data being received from the client. Hence, steps330-340 and step 345 may describe processes that are performedsimultaneously rather than in lock step.

A determination is made as to whether a close file request is receivedby the proxy server from the client at step 350. Once the last file orfile portion is received by the proxy server, the close file requestwill be received by the proxy server from the client. If the close filerequest has not been received from the client, at least one additionalfile or file portion need to be sent and the method continues to step330. If the close file request has been received, a close fileacknowledgment is sent to client 110 from proxy server 130 at step 355.The close file acknowledgment will allow the client to end the filetransfer session and be utilized for different purposes.

The proxy server continues to transmit files or file portions to server150 as long as there is file data in the proxy file buffer. Once all thefile data has been sent to the client, a close file message is sent tothe server at step 360. The user session with the server is then closedat step 365. Hence, the user session with the client will be closedbefore the user session with the server is closed.

FIG. 4 is a method identifying a suitable file for background filetransfer. The method of FIG. 4 provides more detail for step 320 of themethod of FIG. 3.

Steps 410-440 relate to a list of background transfer preferences to besatisfied. The steps in the method of FIG. 4 set forth the preferencesin an exemplary order for purposes of discussion. A background filetransfer system may not require each and every preference associatedwith steps 410-440 to be determined in the affirmative in order toproceed with a background file transfer. For example, a proxyapplication may require only one preference be satisfied, or that aparticular combination of preferences be satisfied.

A determination is made as to whether the request to perform abackground file transfer involves a server, share or folder that matchesa background file preference at step 410. If the server, share or folderin the request does not match that specified in the preference, then themethod continues to step 460.

A determination is made as to whether the request to perform abackground file transfer involves a file extension that matches abackground file preference at step 420. If the file extension in therequest does not match that specified in the preference, then the methodcontinues to step 460.

A determination is made as to whether the request to perform abackground file transfer involves a filename that matches a backgroundfile preference at step 430. If the filename in the request does notmatch that specified in the preference, then the method continues tostep 460.

A determination is made as to whether the request to perform abackground file transfer involves an IO operation pattern match thatmatches a background file preference at step 440. If the IO operationpattern match in the request does not match that specified in thepreference, then the method continues to step 460.

If the request complies with the preferences at steps 410-440, the fileis suitable for background transfer at step 450. If the request does notcomply with the preferences in any of steps 410-440, the file is notsuitable for background transfer at step 460.

FIG. 5 is a method for managing a background file transfer. The methodof FIG. 5 provides details for handling user input through anadministrative interface provided by a proxy application. The interfacemay be utilized for initial configuration as well as during backgroundfile transfer performed at step 330 of the method of FIG. 3. Theinterface may provide a progress of the background transfer to the userat step 310. A determination may be made as to whether the interfacereceives input from a user to open, re-open, delete or rename the fileat step 315. If the input is received, denied response to the clientrequest may be spoofed at step 320, perform some other appropriateaction, and the operation of the method continues to step 325. In someinstances, the administrative interface may allow a file open formetadata access and deletion but prohibit rename, write and readoperations.

Some requests may be selectively permitted. The intent of the user maybe inferred from the requests and mapped into a suitable operation onthe spooling system. For example, “delete” attempts may abort a transferand delete the partially transferred file, while open attempts may bepermitted but use of the file handle may be restricted to reading metadata (file date, size, ownership, etc) and denying reading/writing orrenaming the file.

If the request is not received, a determination is made as to whetherinput is received from a user to delete the background file transfer atstep 325. If no abort request is received, progress of the transfercontinues to be provided through the interface until the transfer iscomplete. If the abort request is received, the file transfer is stoppedand any portion of the batch of files that was transferred to server 150from proxy server 130 is deleted at step 325.

FIG. 6 is a block diagram of an exemplary computing system forimplementing the present technology. System 600 of FIG. 6 may beimplemented in the contexts of the likes of client computer 110, proxyserver 130, and server 150. The computing system 600 of FIG. 6 includesone or more processors 610 and memory 620. Main memory 620 stores, inpart, instructions and data for execution by processor 610. Main memory620 can store the executable code when in operation. The system 600 ofFIG. 6 further includes a mass storage device 630, portable storagemedium drive(s) 640, output devices 650, user input devices 660, agraphics display 670, and peripheral devices 680.

The components shown in FIG. 6 are depicted as being connected via asingle bus 690. However, the components may be connected through one ormore data transport means. For example, processor unit 610 and mainmemory 620 may be connected via a local microprocessor bus, and the massstorage device 630, peripheral device(s) 680, portable storage device640, and display system 670 may be connected via one or moreinput/output (I/O) buses.

Mass storage device 630, which may be implemented with a magnetic diskdrive or an optical disk drive, is a non-volatile storage device forstoring data and instructions for use by processor unit 610. Massstorage device 630 can store the system software for implementingembodiments of the present invention for purposes of loading thatsoftware into main memory 620.

Portable storage device 640 operates in conjunction with a portablenon-volatile storage medium, such as a floppy disk, compact disk orDigital video disc, to input and output data and code to and from thecomputer system 600 of FIG. 6. The system software for implementingembodiments of the present invention may be stored on such a portablemedium and input to the computer system 600 via the portable storagedevice 640.

Input devices 660 provide a portion of a user interface. Input devices660 may include an alpha-numeric keypad, such as a keyboard, forinputting alpha-numeric and other information, or a pointing device,such as a mouse, a trackball, stylus, or cursor direction keys.Additionally, the system 600 as shown in FIG. 6 includes output devices650. Examples of suitable output devices include speakers, printers,network interfaces, and monitors.

Display system 670 may include a liquid crystal display (LCD) or othersuitable display device. Display system 670 receives textual andgraphical information, and processes the information for output to thedisplay device.

Peripherals 680 may include any type of computer support device to addadditional functionality to the computer system. For example, peripheraldevice(s) 680 may include a modem or a router.

The components contained in the computer system 600 of FIG. 6 are thosetypically found in computer systems that may be suitable for use withembodiments of the present invention and are intended to represent abroad category of such computer components that are well known in theart. Thus, the computer system 600 of FIG. 6 can be a personal computer,hand held computing device, telephone, mobile computing device,workstation, server, minicomputer, mainframe computer, or any othercomputing device. As such, the computer system 600 of FIG. 6 may includeadditional components, such as an LED touch screen, one or moreantennas, radios, and other circuitry and software for wirelesscommunication, microphones, speakers, and other components. The computercan also include different bus configurations, networked platforms,multi-processor platforms, etc. Various operating systems can be usedincluding Unix, Linux, Windows, Macintosh OS, Android, and othersuitable operating systems.

The foregoing detailed description of the technology herein has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the technology to the precise formdisclosed. Many modifications and variations are possible in light ofthe above teaching. The described embodiments were chosen in order tobest explain the principles of the technology and its practicalapplication to thereby enable others skilled in the art to best utilizethe technology in various embodiments and with various modifications asare suited to the particular use contemplated. It is intended that thescope of the technology be defined by the claims appended hereto.

What is claimed is:
 1. A method for transferring files, comprising:receiving a first file portion from a client by a first server, thefirst file portion intended for a second server; transmitting a dataacknowledgment to the client from first server; receiving a second fileportion from the client; transmitting an indication that transfer iscomplete to the client by the first server; transmitting the first fileportion and second file portion to the second server; and transmitting aclose file request to the second server from the first server.